Skip to content

Add scripts to allow addons from personal repos to be synchronized with Crowdin#1

Open
nvdaes wants to merge 119 commits intonvaccess:masterfrom
nvdaes:l10n
Open

Add scripts to allow addons from personal repos to be synchronized with Crowdin#1
nvdaes wants to merge 119 commits intonvaccess:masterfrom
nvdaes:l10n

Conversation

@nvdaes
Copy link
Copy Markdown

@nvdaes nvdaes commented Nov 24, 2025

This pull request introduces a complete, automated workflow for synchronizing translations with Crowdin, including new scripts, a scheduled GitHub Actions workflow, and supporting documentation. The main changes add Python and PowerShell scripts for translation status checking and synchronization, a workflow for scheduled and manual Crowdin sync, language mapping configuration, and documentation updates. These improvements enable seamless translation management for add-on projects.

Crowdin Synchronization Workflow and Automation:

  • Added .github/workflows/crowdinL10n.yml to automate translation synchronization with Crowdin, running weekly and on demand. It sets up the environment, downloads required tools, and triggers the sync process.
  • Introduced .github/scripts/crowdinSync.ps1, a comprehensive PowerShell script that updates source files, uploads them to Crowdin, exports translations, evaluates translation quality, and commits updates back to the repository.

Translation Quality and Language Mapping Utilities:

  • Added .github/scripts/checkTranslation.py, a Python script that checks translation progress for a given file and language using the Crowdin API, supporting quality thresholds for importing translations.
  • Added .github/scripts/languageMappings.json to map local language codes to Crowdin-compatible codes, ensuring correct language matching during synchronization.
  • Added .github/scripts/setOutputs.py, a Python script to extract and expose the add-on ID for use in workflows.

Configuration and Documentation:

  • Updated pyproject.toml to include new scripts in the exclude list for packaging and removed the unused requests dependency. [1] [2]
  • Updated readme.md with detailed instructions on setting up and using the Crowdin translation workflow, including project setup and required secrets.
  • Added .python-version specifying Python 3.13 for consistent workflow environments.

These changes collectively provide a robust, maintainable infrastructure for managing translations in add-on projects with Crowdin.

Comment thread pyproject.toml
strictSetInference = true

# Compliant rules
reportAbstractUsage = true
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's probably better to keep these rules than dropping to NVDA's standard

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Nov 28, 2025

Purpose

Add-on authors may wish to help translators use Crowdin, the same framework where they translate NVDA. to translate messages and documentation for maintained add-ons:

Other details

  • Pot file are created/updated, and uploaded to a Crowdin project.
  • The readme.md file is converted to xliff and uploaded to a Crowdin project.
  • Po and xfiles are translated.
  • Translated files are downloaded and processed to be copied to locale/langCode/LC_MESSAGES/nvda.po, and doc/langCode/readme.md, in the addon folder.
    Authors need to store a Crowdin token with permissions to upload files to the Crowdin project as a repository secret.

Development approach

A workflow (GitHub Actions), and several scripts (Python and Powershell), as well as a json file with language mappings have been added.
Also NV Access l10nUtil.exe is used to synchronize files between the add-on and Crowdin, and to convert between xliff and markdown formats.

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Nov 28, 2025

I've tested that all check pass using this pyproject.toml file on this PR:

nvdaes/translateNvdaAddonsWithCrowdin#11

I use precommit, CodeQL and a workflow to check that all translatable messages have comments for translators.

I'll try to use the cache action to cache some add-on metadata like its id, and also hashfiles from l10nSources (taking the value of buildVars.py), and the hasf¡hfile of the readme.md, to determine if pot and xliff files should be updated.

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Nov 30, 2025

Export translations to Crowdin running the workflow with update=False works properly:

https://github.com/nvdaes/translateNvdaAddonsWithCrowdin/actions/runs/19802210157

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Nov 30, 2025

This time, updatexLiff is failing. Seems that adding blank lines to readme may cause problems:

https://github.com/nvdaes/translateNvdaAddonsWithCrowdin/actions/runs/19802391926/job/56731562709

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Nov 30, 2025

If someone can help with this issue when update xliff, I'll be grateful.
I think that this is one of the bugest problems with xliff files. Sometimes sel lines are None and they don't have a strip method. I don't know if this should be also improved in NVDA
cc: @seanbudd

@seanbudd
Copy link
Copy Markdown
Member

seanbudd commented Dec 1, 2025

It might be easier to avoid xliff and just translate the markdown files directly. This won't support diffs very well but worth experimenting with

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Dec 1, 2025

@seanbudd wrote:

It might be easier to avoid xliff and just translate the markdown files directly. This won't support diffs very well but worth experimenting with

OK.

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Dec 3, 2025

@CyrilleB79, you were interested in this framework. If you want, feel free to see how the translateNvdaAddonsWithCrowdin.md can be translated in the project. Using xliff files is causing problems, as mentioned, and we are experimenting uploading md files instead.

@abdel792
Copy link
Copy Markdown

Hi @nvdaes,

That’s a great point. The langCodes.py script was originally included to handle specific edge cases where Crowdin language codes and NVDA local directory names might differ (such as handling underscores vs dashes, or specific regional variants).

However, you are right: if our languageMappings.json and NVDA lang directories correspond with _addonL10n subfolders structure, we could definitely simplify this.

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026 via email

- Updated crowdinSync.ps1 to use $langCode directly for local paths, as it matches NVDA's structure[cite: 2].
- Removed the intermediate $localLangDir variable for better readability[cite: 2].
- Deleted the now obsolete langCodes.py script[cite: 2].
- Improved internal documentation regarding language code handling[cite: 2].

Special thanks to @nvdaes for suggesting this optimization!
@abdel792
Copy link
Copy Markdown

Hi @nvdaes,

I've just implemented the simplification you suggested!

Since the folder names from Crowdin already match the NVDA directory structure, I've updated crowdinSync.ps1 to use $langCode directly for all local paths[cite: 2]. I have also removed the langCodes.py script entirely, as it is no longer needed[cite: 2].

This makes the workflow leaner and faster by avoiding unnecessary Python calls. Thanks again for the great suggestion! ☺

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Thanks for the tips, @nvdaes!

I'll make sure to add the license header and my name to the scripts to comply with NV Access requirements.

Regarding Pyright, I'll do my best to keep the code clean and follow NVDA standards so the PR is as professional as possible.

I'm glad to hear we are almost there! ☺

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Regarding the author name and license, I'll add them to be fully compliant with NV Access rules. It's not a priority for me personally, but I want to make sure the PR meets all the formal requirements for a smooth integration. ☺

@abdel792
Copy link
Copy Markdown

Hi @nvdaes,

I had a small follow-up regarding the script headers.

Would you mind adding the copyright information? I think you're more familiar with that than I am 🙂

Also, for crowdinSync.ps1, it would make sense to list both of us as authors, since you contributed to it as well.

Let me know what you think!

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026 via email

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026 via email

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Thanks @nvdaes,

That's very kind of you!

You're right, adding the type annotations and docstrings following NVDA standards will definitely help with Pyright and the final review.

- Renamed functions to camelCase (findFileId, getScoreFromApi) to match NVDA standards[cite: 1].
- Added Python type annotations to improve Pyright compatibility[cite: 1].
- Included detailed docstrings using @param and @type tags for documentation[cite: 1].
- Refined the main function entry point with proper type hinting[cite: 1].

Special thanks to @nvdaes for the guidance on code standards and for offering to handle the license headers!
@abdel792
Copy link
Copy Markdown

Hi again @nvdaes,

I've just pushed the latest updates to checkTranslation.py!

  • NVDA Standards: I have renamed the functions to camelCase (findFileId, getScoreFromApi) to align with NVDA's coding style.
  • Documentation & Types: I've added type annotations to satisfy Pyright and included docstrings using the @param and @type format for better maintainability[cite: 1].

I'll let you add the copyright headers as you kindly offered. I think the script logic and structure are now fully ready for the final review. Thank you again for your precious help! ☺

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026 via email

@abdel792
Copy link
Copy Markdown

Hi @nvdaes,

Thank you for these final clarifications!

Please go ahead and add the copyrights (including NV Access as you suggested) and update the docstrings to the :param: format. I'm also happy for you to adjust the remaining variables to camelCase (like projectId) to ensure everything is perfectly aligned with NVDA standards.

I really appreciate you taking care of these last formal details. Once you're done, I think we're all set for the final integration! ☺

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Hi @nvdaes,

Thanks for the updates! I noticed a small bug in the last commit that triggers an API ERROR: 'language_id'.

It seems that when converting variables to camelCase, the dictionary key for the Crowdin API response was changed to language_id (line 98).

However, the Crowdin API returns this specific key as languageId.

I've tested it by changing it back to langApi = item["data"]["languageId"] and it works perfectly again.

Do you want me to push the fix or will you handle it?

Thanks! ☺

The recent refactoring to camelCase accidentally changed the dictionary key 'languageId' to 'language_id' when accessing the API response data. Since the Crowdin API strictly returns 'languageId', this caused a KeyError.

- Restored 'languageId' in getScoreFromApi function to ensure proper score retrieval[cite: 2].
- Verified that other internal camelCase variables remain unchanged to satisfy NVDA coding standards[cite: 2].
@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026

@abdel792 wrote:

Do you want me to push the fix or will you handle it?

Please fix it yourself. Do you want to test it in one of your add-ons, or should I perform the final test?

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Hi @nvdaes,

I've just pushed a fix for the API ERROR: 'language_id'.

During the camelCase refactoring, the dictionary key for the Crowdin API response was changed to language_id (line 98). However, since this is a key returned directly by the Crowdin API, it must remain languageId to be correctly mapped.

I have:

  • Restored item["data"]["languageId"] to fix the KeyError.
  • Kept the internal variable name as langApi (camelCase) to respect NVDA standards[cite: 1].

The script is now working perfectly again for all file types (.po, .md, and .xliff). Sorry for missing that during the initial review! ☺

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026

Thanks so much. I'l perform the final test.

@abdel792
Copy link
Copy Markdown

Hi @nvdaes,

I've just pushed the fix for languageId.

I am currently testing the script directly on my PC using PowerShell, as I mentioned before. I'm iterating through my local add-on directories and querying the Crowdin API for each one to verify the scores. Everything is working as expected now!

You can perform a final test on your side if you wish, but from my end, the script is fully functional and ready for the final review. ☺

@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026

I'll perform a final test to show publicly that it works. Thanks for your work!

@nvdaes nvdaes marked this pull request as ready for review April 29, 2026 16:26
@nvdaes
Copy link
Copy Markdown
Author

nvdaes commented Apr 29, 2026

@abdel792 , I've run pre-commit to fix chekcs.
@seanbudd I think this is ready for review.
Please see this test:

https://github.com/nvdaes/readFeeds/actions/runs/25120306831/job/73618870755

@abdel792
Copy link
Copy Markdown

abdel792 commented Apr 29, 2026

Thanks @nvdaes for running the pre-commit fixes! Glad to see the checks are passing now. Looking forward to @seanbudd's review.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 12 changed files in this pull request and generated 8 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +22 to +24
Copy-Item "$addonId.xliff" $tempXliff -Force
Write-Host "DEBUG: Updating XLIFF source based on readme.md..."
uv run .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A temp file is created for the previous XLIFF (GetTempFileName()), but it’s never removed. Wrap the update in a try/finally (or remove the temp file after use) to avoid leaking temp files on repeated runs.

Suggested change
Copy-Item "$addonId.xliff" $tempXliff -Force
Write-Host "DEBUG: Updating XLIFF source based on readme.md..."
uv run .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
try {
Copy-Item "$addonId.xliff" $tempXliff -Force
Write-Host "DEBUG: Updating XLIFF source based on readme.md..."
uv run .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
} finally {
if (Test-Path $tempXliff) {
Remove-Item $tempXliff -Force
}
}

Copilot uses AI. Check for mistakes.
Comment thread readme.md
Comment on lines +188 to +191
#### 2. GitHub Secrets
To allow the workflows to communicate with Crowdin, you must add the following secrets to your GitHub repository (`Settings > Secrets and variables > Actions`):
* `crowdinAuthToken`: Paste your Crowdin API token here.

Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documented secret name (crowdinAuthToken) doesn’t match what the workflow actually reads (secrets.CROWDIN_TOKEN). As written, following the README will leave the workflow without credentials. Align the README and workflow on a single secret name (either change the workflow to use secrets.crowdinAuthToken or update the documentation to instruct users to create CROWDIN_TOKEN).

Copilot uses AI. Check for mistakes.
Comment thread readme.md
Comment on lines +194 to +195
* **Workflows:** `.github/workflows/crowdinL10n.yml**
* **Scripts:** The `.github/scripts/` folder containing `checkTranslation.py`, `langCodes.py`, `languageMappings.json`, `setOutputs.py`, and `crowdinSync.ps1`.
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow bullet has broken Markdown (missing closing backtick / bold marker), and the scripts list references langCodes.py which isn’t present in .github/scripts in this PR (while markdownTranslate.py is present but not mentioned). Update this section so the file list matches what’s actually shipped and renders correctly.

Suggested change
* **Workflows:** `.github/workflows/crowdinL10n.yml**
* **Scripts:** The `.github/scripts/` folder containing `checkTranslation.py`, `langCodes.py`, `languageMappings.json`, `setOutputs.py`, and `crowdinSync.ps1`.
* **Workflows:** `.github/workflows/crowdinL10n.yml`
* **Scripts:** The `.github/scripts/` folder containing `checkTranslation.py`, `markdownTranslate.py`, `languageMappings.json`, `setOutputs.py`, and `crowdinSync.ps1`.

Copilot uses AI. Check for mistakes.
Comment thread pyproject.toml
# paths are relative to the configuration file.
".github/scripts/markdownTranslate.py",
".github/scripts/checkTranslation.py",
".github/scripts/langCodes.py",
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pyright’s exclude list includes .github/scripts/langCodes.py, but that file doesn’t exist in this PR. Either add the missing script or remove the stale exclude entry to avoid confusion about expected tooling/scripts in the template.

Suggested change
".github/scripts/langCodes.py",

Copilot uses AI. Check for mistakes.
cancel-in-progress: true

env:
crowdinAuthToken: ${{ secrets.CROWDIN_TOKEN }}
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow sets crowdinAuthToken from secrets.CROWDIN_TOKEN, but the repository documentation instructs users to create a crowdinAuthToken secret. This mismatch will cause authentication to fail for users following the README. Use a single secret name consistently (e.g., secrets.crowdinAuthToken) and update docs accordingly.

Suggested change
crowdinAuthToken: ${{ secrets.CROWDIN_TOKEN }}
crowdinAuthToken: ${{ secrets.crowdinAuthToken }}

Copilot uses AI. Check for mistakes.
- name: Get add-on info
id: getAddonInfo
shell: pwsh
run: uv run ./.github/scripts/setOutputs.py
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uv run ./.github/scripts/setOutputs.py relies on executing a .py file directly. Since setOutputs.py has no shebang and uv run executes commands without a shell, this will typically fail on Windows with “not a valid Win32 application”. Invoke it explicitly via Python (e.g., uv run python ./.github/scripts/setOutputs.py).

Suggested change
run: uv run ./.github/scripts/setOutputs.py
run: uv run python ./.github/scripts/setOutputs.py

Copilot uses AI. Check for mistakes.
Comment on lines +8 to +12
$addonId = $env:ADDON_ID.Trim()
if (-not $addonId) {
Write-Error "Failed to get addon ID."
exit 1
}
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$env:ADDON_ID.Trim() will throw if ADDON_ID isn’t set (null), so the script can terminate before reaching the intended error message. Read the env var first and only call .Trim() after verifying it’s non-null/non-empty (or use [string]::IsNullOrWhiteSpace).

Suggested change
$addonId = $env:ADDON_ID.Trim()
if (-not $addonId) {
Write-Error "Failed to get addon ID."
exit 1
}
$rawAddonId = $env:ADDON_ID
if ([string]::IsNullOrWhiteSpace($rawAddonId)) {
Write-Error "Failed to get addon ID."
exit 1
}
$addonId = $rawAddonId.Trim()

Copilot uses AI. Check for mistakes.
Comment on lines +24 to +27
uv run .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
} else {
Write-Host "DEBUG: XLIFF template not found. Creating new one from readme.md..."
uv run .github/scripts/markdownTranslate.py generateXliff -m $mdFile -o $xliffFile
Copy link

Copilot AI Apr 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

uv run .github/scripts/markdownTranslate.py ... attempts to execute the Python file as a program. Since markdownTranslate.py isn’t an executable with a shebang, this is likely to fail under uv run. Call it explicitly via Python (e.g., uv run python .github/scripts/markdownTranslate.py ...) for both the update and generate commands.

Suggested change
uv run .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
} else {
Write-Host "DEBUG: XLIFF template not found. Creating new one from readme.md..."
uv run .github/scripts/markdownTranslate.py generateXliff -m $mdFile -o $xliffFile
uv run python .github/scripts/markdownTranslate.py updateXliff -m $mdFile -x $tempXliff -o $xliffFile
} else {
Write-Host "DEBUG: XLIFF template not found. Creating new one from readme.md..."
uv run python .github/scripts/markdownTranslate.py generateXliff -m $mdFile -o $xliffFile

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants